Papers with Quantitative analysis
MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
Chia-Yuan Chang, Zhimeng Jiang, Vineeth Rakesh, Menghai Pan, Chin-Chia Michael Yeh, Guanchu Wang, Mingzhi Hu, Zhichao Xu, Yan Zheng, Mahashweta Das, Na Zou
| Challenge: | Existing RAG systems struggle with the quality of retrieval documents, causing performance degradation and reducing performance. |
| Approach: | They propose a training-free RAG framework that leverages multiple LLM agents to collaboratively filter and score retrieved documents. |
| Outcome: | The proposed framework outperforms existing RAG frameworks in QA benchmarks and shows superior answer consistency and answer accuracy over baseline methods. |
Building a Japanese Document-Level Relation Extraction Dataset Assisted by Cross-Lingual Transfer (2024.lrec-main)
Copied to clipboard
| Challenge: | Document-level Relation Extraction (DocRE) is the task of extracting all semantic relationships from a document. |
| Approach: | They propose to transfer an English document to Japanese to promote DocRE in other languages. |
| Outcome: | The proposed model reduces the human edit steps by 50% compared with the previous approach. |
Neural network embeddings recover value dimensions from psychometric survey items on par with human data (2026.findings-eacl)
Copied to clipboard
| Challenge: | Embedings from large language models can recover structure of human values . quantitative analysis reveals that SQuID addresses the challenge of obtaining negative correlations between dimensions without domain-specific fine-tuning or training data reannotation. |
| Approach: | They propose to use questionnaire item embeddings to recover human values from PVQ-RR . their results have implications for psychometrics and social science research . |
| Outcome: | The proposed method explains 55% variance in dimension-dimension similarities compared to human data. |
Auditing Deep Learning processes through Kernel-based Explanatory Models (D19-1)
Copied to clipboard
| Challenge: | Existing nonlinearity of deep learning models can be a major drawback . ethical accountability of such systems is becoming a crucial issue . |
| Approach: | They propose to use Layerwise Relevance Propagation to trace back connections between linguistic properties of input instances and system decisions. |
| Outcome: | The proposed model evaluates the transparency and coherence of analogy-based explanations modeling an audit stage for the system. |